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THE MAILING DATE OF THIS COMMUNICATION. 
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DETAILED ACTION 



1 . This action is responsive to amendment filed 3/24/03. 

2. Claims 1-24, 29-39, 43-63 are pending in the application. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1-13, 16, 24, 29, 31-33, 39 and 43-63 are rejected under 35 U.S.C. 

103(a) as being unpatentable Zhang et al. (6,608,923) in view of Ayache et al. 

(6,175,648). 

Claim 1, Zhang et al. (6,608,923) discloses obtaining two images of similar image 
information (212, 214 of fig. 2; fig. 3, #314) from two uncalibrated sources (col. 6, lines 
58-60); superimposing lines (pairs of corresponding epipolar lines) to rectify the two 
images (col. 7, lines 48-66; fig. 5); using the rectified images to form 3D (col. 2, lines 25- 
37) by forming a disparity map of 3D. Zhang et al. does not teach forming of 3D by 
using a disparity map; however Ayache et al. teaches using the rectified images to form 
3D by forming a disparity map of 3D (col. 4, line 9 through col. 7, line 29). It would have 
been obvious to one of ordinary skill in the art at the time the invention was made to 
incorporate the disparity map as taught by Ayache into the method for rectifying images 
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of Zhang for forming 3D image from 2D similar images, because it would provide a 
process for producing cartographic data in 3D from 2D images of the scene (col. 2, lines 
35-37). 

Claims 2-4, 9-13,16, 24, Ayache et al. teaches disparity map indicating an 
estimate of 3D surface and information of the estimate (col, 2, lines 40-44; col. 4, lines 
12-17); manually establishing matching parts (col. 4, lines 32-34, lines 47-49); 
identifying parts in the image (col. 8, lines 40-44), using automatically rectified images to 
form 3D information (col. 5, lines 3-14); forming a disparity surface indicative of 3D 
information of the image (col. 8, lines 65-67); a difference between coordinates of 
matching pixels (col. 9, lines 8-10); a variable denoting a degree of similarity between 
pixels (col. 8, lines 55-58), a variable of error in 3D surface (a degree of disparity which 
is inconsistent with the disparities on all the other curves; col. 9, lines 6-7). It would have 
been obvious to one of ordinary skill in the art at the time the invention was made to 
incorporate the disparity map, estimating, matching as taught by Ayache for forming 3D 
image from 2D similar images, because it would provide a process for producing 
cartographic data in 3D from 2D images of the scene (col. 2, lines 35-37). 

Claims 5-7, Zhang (6,608,923) discloses defining images in terms of epipolar 
geometry (col. 5, lines 29-32; col. 6, lines 44-49), aligning the images in the epipolar 
geometry (col. 14, lines 51-62); aligning specified reference lines which include lines 
passing through manually-obtained image parts (col. 12, lines 34-53). 
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Claim 8, Zhang (6,608,923) discloses finding an average of end points of two 
different reference lines (a pair of epipolar lines), forming a line through an averaged 
part (col. 6, lines 30-43; figs. 4 and 5). 

Claims 29, 31, 32, 33, Zhang (6,608,923) discloses obtaining first and second 
images of the same object (fig. 2; #210, #212, #214); identifying objects in the first and 
second images, forming lines which intersect the objects; aligning the lines in an 
epipolar geometry (figs. 4 and 5); using the first and second images with the aligned 
lines to form 3D information (figs. 8-11); identifying seed voxel (3D corresponding pixel 
information u,v that is unique) for different parts of 3D surface with a high probability of 
being correct 3D which is greater than a specified threshold (0.99 or equals to one); 
(col. 6, lines 2-9). Zhang does not teach identifying a seed voxel; however, Zhang 
teaches 3D object, in the image, points and lines are represented by 3D column 
vectors, where the points are uniquely identified by u, v and w (col. 5, line 59 through 
col. 6, line 9), and could be a voxel when pixel is identified in the 3D space data or 
volume data. Therefore, it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to include a seed voxel, because rectification of each 
image is produced with minimum distortion (col. 2, lines 36-37). 

Claim 39, the rationale provided in the rejection of claims 9 and 29 is 
incorporated herein. 

Claims 43 and 44, Zhang (6,608,923) teaches an image from two uncalibrated 
cameras to obtain 3D information (col. 6, lines 18-22, lines 59-60); rectifying the image 
(col. 2, lines 36-37) to form coplanar images with scan lines are horizontally parallel; 
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identifying points, scan lines which pass through the points forming horizontally parallel 
scan lines (col. 1 , lines 38-50). Claim 45, Zhang does not teach disparity map; however, 
Ayache teaches disparity map (col. 7, lines18-30). It would have been obvious to one of 
ordinary skill in the art at the time the invention was made to incorporate the disparity 
map as taught by Ayache into the method for rectifying images of Zhang for forming 3D 
image from 2D similar images, because it would provide a process for producing 3D 
image from 2D image of a scene (col. 1 , lines 5-7). 

Claims 46-63, the rationale provided in the rejection of claims 1 , 3, 5-8, 29, 43, 44 
above is incorporated herein. 

5. Claims 19-23 are rejected under 35 U.S.C. 103(a) as being unpatentable Zhang 
et al. (6,608,923) in view of Ayache et al. (6,175,648) and further in view of Sawhney 
et al. (6,571,024). 

Claims 19-23, Zhang does not teach Euclidean points; however, Sawhney et al. 
teaches converting the volume to Euclidean points (col. 5, lines 53-55); projecting a 
reconstruction of the volume, reconstructing Euclidean points from the projective 
reconstruction (col. 6, lines 25-36); transforming an origin of coordinate system to an 
origin of the images (image transformation; fig. 18, # 804); input a parameter (focal 
length; adjust the parameter (refining parameter; fig. 18, # 806) to approximate a proper 
Euclidean reconstruction (col. 5, lines 37-61). It would have been obvious to one of 
ordinary skill in the art at the time the invention was made to incorporate the Euclidean 
reconstructing as taught by Sawhney into the method for rectifying images of Zhang for 
forming 3D image from 2D similar images, because it would provide a method of 
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camera pose and scene geometric information for each frame of a video sequence (col. 
2, lines 2-3). 

6. Claims 14-16, 24, 30, 34-38 are rejected under 35 U.S.C. 103(a) as being 
unpatentable Zhang et al. (6,608,923) in view of Ayache et al. (6,175,648), and further 
in view of VanEssen et al. (6,591 ,004). 

Claims 14, 15, Zhang does not teach tracing voxels; however, VanEssen et al. 
teaches tracing voxels in a multiresolution at coarsest level and a more detailed level 
(col. 25, lines 42-45; col. 26, lines 55-67); selecting the seed voxel using a winner take 
which has a maximum correlation value; identifying seed voxels which represent 
incorrect matches, removing the seeds after tracing (col. 19, lines 53-67). Claim 16, 24, 
Zhang does not teach a seed voxel; however, VanEssen et al. teaches selecting a seed 
voxel by finding uniqueness (col. 44, line 65 through col. 45, line 23); dividing the 
surface into parts (segmented), the seed voxels are respectively for parts (col. 22, lines 
28-36). Claim 30, 34-38, VanEssen et al. teaches forming the surface map by 
propagating from the seed voxel; determining intersection between two voxels at 
different parts; determining probability of two voxels finding neighbors for voxels one by 
one);(col. 25, lines 49-55), and selecting the higher probability as the voxel to use 
selecting a seed voxel that the best matches the corresponding pixel information and 
has a probability of being correct which is greater than a specified threshold (col. 26, 
line 64 through col. 27, line 5); identifying voxels represent incorrect matches and 
removing incorrect matching voxels (col. 23, lines 27-40). It would have been obvious to 
one of ordinary skill in the art at the time the invention was made to incorporate the 
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tracing voxels and selecting voxels as taught by Van Essen into the method for rectifying 
images of Zhang for forming 3D image from 2D similar images, because it would 
provide a method for reconstructing surfaces and analyzing surface volume 
representations of the shape of an object corresponding to image data, in which the 
object has been modeled as one or more physically distinct compartments (col. 2, lines 
53-57). 

7. Claims 17, 18 are rejected under 35 U.S.C. 103(a) as being unpatentable Zhang 
et al. (6,608,923) in view of Ayache et al. (6,175,648), and further in view of VanEssen 
et al. (6,591,004) and Szeliski et al. (5,917,937). 

Claims 17 and 18, Zhang does not teach using a winner take which has a 
maximum correlation value; however, Szeliski et al. teaches winning cell (col. 9, lines 
40-45). It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate the seed voxel and winning cell as taught by Szeliski 
into the rectifying images of Zhang's method for forming 3D images, because it would 
provide a stereo matching method simultaneously recovers disparities from input 
images to reconstruct 3D surface (abstract). 

8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Kimbinh Nguyen whose telephone number is (703) 
305-9683. The examiner can normally be reached (Monday- Thursday from 7:00 AM 
to 4:30 PM and alternate Fridays from 7:00 AM to 3:30 PM). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mark Zimmerman, can be reached at (703) 305-9798. 
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Any response to this action should be mailed to: 

Commissioner of Patents and Trademarks 

Washington, D.C. 20231 
Or faxed to: 

(703) 872-9314 (for Technology Center 2600 only) 

Hand-delivered responses should be brought to Crystal Part II, 2121 Crystal Drive, 
Arlington, VA, Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the Technology Center 2600 Customer Service Office 
whose telephone number is (703) 306-0377. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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Kimbinh Nguyen 

Patent Examiner AU 2671 



